Deformable cardiac surface tracking by adaptive estimation algorithms.

This study presents a particle filter based framework to track cardiac surface from a time sequence of single magnetic resonance imaging (MRI) slices with the future goal of utilizing the presented framework for interventional cardiovascular magnetic resonance procedures, which rely on the accurate and online tracking of the cardiac surface from MRI data. The framework exploits a low-order parametric deformable model of the cardiac surface. A stochastic dynamic system represents the cardiac surface motion. Deformable models are employed to introduce shape prior to control the degree of the deformations. Adaptive filters are used to model complex cardiac motion in the dynamic model of the system. Particle filters are utilized to recursively estimate the current state of the system over time. The proposed method is applied to recover biventricular deformations and validated with a numerical phantom and multiple real cardiac MRI datasets. The algorithm is evaluated with multiple experiments using fixed and varying image slice planes at each time step. For the real cardiac MRI datasets, the average root-mean-square tracking errors of 2.61 mm and 3.42 mm are reported respectively for the fixed and varying image slice planes. This work serves as a proof-of-concept study for modeling and tracking the cardiac surface deformations via a low-order probabilistic model with the future goal of utilizing this method for the targeted interventional cardiac procedures under MR image guidance. For the real cardiac MRI datasets, the presented method was able to track the points-of-interests located on different sections of the cardiac surface within a precision of 3 pixels. The analyses show that the use of deformable cardiac surface tracking algorithm can pave the way for performing precise targeted intracardiac ablation procedures under MRI guidance. The main contributions of this work are twofold. First, it presents a framework for the tracking of whole cardiac surface from a time sequence of single image slices. Second, it employs adaptive filters to incorporate motion information in the tracking of nonrigid cardiac surface motion for temporal coherence.

State of the Art and Future Opportunities in MRI-Guided Robot-Assisted Surgery and Interventions.

Magnetic resonance imaging (MRI) can provide high-quality 3-D visualization of target anatomy, surrounding tissue, and instrumentation, but there are significant challenges in harnessing it for effectively guiding interventional procedures. Challenges include the strong static magnetic field, rapidly switching magnetic field gradients, high-power radio frequency pulses, sensitivity to electrical noise, and constrained space to operate within the bore of the scanner. MRI has a number of advantages over other medical imaging modalities, including no ionizing radiation, excellent soft-tissue contrast that allows for visualization of tumors and other features that are not readily visible by other modalities, true 3-D imaging capabilities, including the ability to image arbitrary scan plane geometry or perform volumetric imaging, and capability for multimodality sensing, including diffusion, dynamic contrast, blood flow, blood oxygenation, temperature, and tracking of biomarkers. The use of robotic assistants within the MRI bore, alongside the patient during imaging, enables intraoperative MR imaging (iMRI) to guide a surgical intervention in a closed-loop fashion that can include tracking of tissue deformation and target motion, localization of instrumentation, and monitoring of therapy delivery. With the ever-expanding clinical use of MRI, MRI-compatible robotic systems have been heralded as a new approach to assist interventional procedures to allow physicians to treat patients more accurately and effectively. Deploying robotic systems inside the bore synergizes the visual capability of MRI and the manipulation capability of robotic assistance, resulting in a closed-loop surgery architecture. This article details the challenges and history of robotic systems intended to operate in an MRI environment and outlines promising clinical applications and associated state-of-the-art MRI-compatible robotic systems and technology for making this possible.

MRI Distortion Correction and Robot-to-MRI Scanner Registration for an MRI-Guided Robotic System.

Magnetic resonance imaging (MRI) guided robotic procedures require safe robotic instrument navigation and precise target localization. This depends on reliable tracking of the instrument from MR images, which requires accurate registration of the robot to the scanner. A novel differential image based robot-to-MRI scanner registration approach is proposed that utilizes a set of active fiducial coils, where background subtraction method is employed for coil detection. In order to use the presented preoperative registration approach jointly with the real-time high speed MRI image acquisition and reconstruction methods in real-time interventional procedures, the effects of the geometric MRI distortion in robot to scanner registration is analyzed using a custom distortion mapping algorithm. The proposed approach is validated by a set of target coils placed within the workspace, employing multi-planar capabilities of the scanner. Registration and validation errors are respectively 2.05 mm and 2.63 mm after the distortion correction showing an improvement of respectively 1.08 mm and 0.14 mm compared to the results without distortion correction.

Localization of Point-of-Interest Positions on Cardiac Surface for Robotic-Assisted Beating Heart Surgery.

One of the critical components of robotic-assisted beating heart surgery is precise localization of a point-of-interest (POI) position on cardiac surface, which needs to be tracked by the robotic instruments. This is challenging as the incoming sensor measurements, from which POI position is localized, might be noisy and incomplete. This paper presents two Bayesian filtering based localization approaches to localize POI position online from sonomicrometer measurements. Specifically, extended Kalman filter (EKF) and particle filter (PF) localization algorithms are explored to estimate the state of POI position. The estimations of upcoming heart motion generated by the generalized adaptive predictor, which is demonstrated in the authors' past work, are also incorporated to generate an improved motion model. The proposed methods are validated with prerecorded in-vivo heart motion data.

A Probabilistic Approach for Contact Stability and Contact Safety Analysis of Robotic Intracardiac Catheter.

The disturbances caused by the blood flow and tissue surface motions are major concerns during the motion planning of an intracardiac robotic catheter. Maintaining a stable and safe contact on the desired ablation point is essential for achieving effective lesions during the ablation procedure. In this paper, a probabilistic formulation of the contact stability and the contact safety for intravascular cardiac catheters under the blood flow and surface motion disturbances is presented. Probabilistic contact stability and contact safety metrics, employing a sample-based representation of the blood flow velocity distribution and the heart motion trajectory, are introduced. Finally, the contact stability and safety for an magnetic resonance imaging-actuated robotic catheter under main pulmonary artery blood flow disturbances and left ventricle surface motion disturbances are analyzed in simulation as example scenarios.

Contact Stability and Contact Safety of a Magnetic Resonance Imaging-Guided Robotic Catheter Under Heart Surface Motion.

Contact force quality is one of the most critical factors for safe and effective lesion formation during catheter based atrial fibrillation ablation procedures. In this paper, the contact stability and contact safety of a novel magnetic resonance imaging (MRI)-actuated robotic cardiac ablation catheter subject to surface motion disturbances are studied. First, a quasi-static contact force optimization algorithm, which calculates the actuation needed to achieve a desired contact force at an instantaneous tissue surface configuration is introduced. This algorithm is then generalized using a least-squares formulation to optimize the contact stability and safety over a prediction horizon for a given estimated heart motion trajectory. Four contact force control schemes are proposed based on these algorithms. The first proposed force control scheme employs instantaneous heart position feedback. The second control scheme applies a constant actuation level using a quasi-periodic heart motion prediction. The third and the last contact force control schemes employ a generalized adaptive filter-based heart motion prediction, where the former uses the predicted instantaneous position feedback, and the latter is a receding horizon controller. The performance of the proposed control schemes is compared and evaluated in a simulation environment.

Analysis of Contact Stability and Contact Safety of a Robotic Intravascular Cardiac Catheter under Blood Flow Disturbances.

This paper studies the contact stability and contact safety of a robotic intravascular cardiac catheter under blood flow disturbances while in contact with tissue surface. A probabilistic blood flow disturbance model, where the blood flow drag forces on the catheter body are approximated using a quasi-static model, is introduced. Using this blood flow disturbance model, probabilistic contact stability and contact safety metrics, employing a sample based representation of the blood flow velocity distribution, are proposed. Finally, the contact stability and contact safety of a MRI-actuated robotic catheter are analyzed using these models in a specific example scenario under left pulmonary inferior vein (LIV) blood flow disturbances.

Contact Stability Analysis of Magnetically-Actuated Robotic Catheter Under Surface Motion.

Contact force quality is one of the most critical factors for safe and effective lesion formation during cardiac ablation. The contact force and contact stability plays important roles in determining the lesion size and creating a gap-free lesion. In this paper, the contact stability of a novel magnetic resonance imaging (MRI)-actuated robotic catheter under tissue surface motion is studied. The robotic catheter is modeled using a pseudo-rigid-body model, and the contact model under surface constraint is provided. Two contact force control schemes to improve the contact stability of the catheter under heart surface motions are proposed and their performance are evaluated in simulation.

Differential Image Based Robot to MRI Scanner Registration with Active Fiducial Markers for an MRI-Guided Robotic Catheter System.

In magnetic resonance imaging (MRI) guided robotic catheter ablation procedures, reliable tracking of the catheter within the MRI scanner is needed to safely navigate the catheter. This requires accurate registration of the catheter to the scanner. This paper presents a differential, multi-slice image-based registration approach utilizing active fiducial coils. The proposed method would be used to preoperatively register the MRI image space with the physical catheter space. In the proposed scheme, the registration is performed with the help of a registration frame, which has a set of embedded electromagnetic coils designed to actively create MRI image artifacts. These coils are detected in the MRI scanner's coordinate system by background subtraction. The detected coil locations in each slice are weighted by the artifact size and then registered to known ground truth coil locations in the catheter's coordinate system via least-squares fitting. The proposed approach is validated by using a set of target coils placed withing the workspace, employing multi-planar capabilities of the MRI scanner. The average registration and validation errors are respectively computed as 1.97 mm and 2.49 mm. The multi-slice approach is also compared to the single-slice method and shown to improve registration and validation by respectively 0.45 mm and 0.66 mm.

Camera-Robot Calibration for the da Vinci® Robotic Surgery System.

The development of autonomous or semi-autonomous surgical robots stands to improve the performance of existing teleoperated equipment, but requires fine hand-eye calibration between the free-moving endoscopic camera and patient-side manipulator arms (PSMs). A novel method of solving this problem for the da Vinci® robotic surgical system and kinematically similar systems is presented. First, a series of image-processing and optical-tracking operations are performed to compute the coordinate transformation between the endoscopic camera view frame and an optical-tracking marker permanently affixed to the camera body. Then, the kinematic properties of the PSM are exploited to compute the coordinate transformation between the kinematic base frame of the PSM and an optical marker permanently affixed thereto. Using these transformations, it is then possible to compute the spatial relationship between the PSM and the endoscopic camera using only one tracker snapshot of the two markers. The effectiveness of this calibration is demonstrated by successfully guiding the PSM end effector to points of interest identified through the camera. Additional tests on a surgical task, namely grasping a surgical needle, are also performed to validate the proposed method. The resulting visually-guided robot positioning accuracy is better than the earlier hand-eye calibration results reported in the literature for the da Vinci® system, while supporting intraoperative update of the calibration and requiring only devices that are already commonly used in the surgical environment.

Jacobian-Based Task-Space Motion Planning for MRI-Actuated Continuum Robots.

Robot-assisted medical interventions, such as robotic catheter ablation, often require the robot to perform tasks on a tissue surface. This paper presents a task-space motion planning method that generates actuation trajectories which steer the end- effector of the MRI-actuated robot along desired trajectories on the surface. The continuum robot is modeled using the pseudo-rigid-body model, where the continuum body of the robot is approximated by rigid links joined by flexible joints. The quasistatic motion model of the robot is formulated as a potential energy minimization problem. The Jacobian of the quasistatic motion model is used in calculating the actuations that steer the tip in the desired directions. The proposed method is validated experimentally in a clinical 3-T MRI scanner.

Real-Time Visual Tracking of Dynamic Surgical Suture Threads.

In order to realize many of the potential benefits associated with robotically assisted minimally invasive surgery, the robot must be more than a remote controlled device. Currently, using a surgical robot can be challenging, fatiguing, and time consuming. Teaching the robot to actively assist surgical tasks, such as suturing, has the potential to vastly improve both patient outlook and the surgeon's efficiency. One obstacle to completing surgical sutures autonomously is the difficulty in tracking surgical suture threads. This paper presents novel stereo image processing algorithms for the detection, initialization, and tracking of a surgical suture thread. A Non Uniform Rational B-Spline (NURBS) curve is used to model a thin, deformable, and dynamic length thread. The NURBS model is initialized and grown from a single selected point located on the thread. The NURBS curve is optimized by minimizing the image matching energy between the projected stereo NURBS image and the segmented thread image. The algorithms are evaluated using suture threads, a calibrated test pattern, and a simulated thread image. Additionally, the accuracy of the algorithms presented are validated as they track a suture thread undergoing translation, deformation, and apparent length changes. All of the tracking is in real-time. Note to Practioners: Abstract-The problem of tracking a surgical suture thread was addressed in this work. Since the suture thread is highly deformable, any tracking algorithm must be robust to intersections, occlusions, knot tying, and length changes. The detection algorithm introduced in this paper is capable of distinguishing different threads when they intersect. The tracking algorithm presented here demonstrate that it is possible, using polynomial curves, to track a suture thread as it deforms, becomes occluded, changes length, and even ties a knot in real time. The detection algorithm can enhance directional thin features while the polynomial curve modeling can track any string like structure. Further integration of the polynomial curve with a feed-forward thread model could improve the stability and robustness of the thread tracking.

Active Localization and Tracking of Needle and Target in Robotic Image-Guided Intervention Systems.

This paper describes a framework of algorithms for the active localization and tracking of flexible needles and targets during image-guided percutaneous interventions. The needle and target configurations are tracked by Bayesian filters employing models of the needle and target motions and measurements of the current system state obtained from an intra-operative imaging system which is controlled by an entropy-minimizing active localization algorithm. Versions of the system were built using particle and unscented Kalman filters and their performance was measured using both simulations and hardware experiments with real magnetic resonance imaging data of needle insertions into gel phantoms. Performance of the localization algorithms is given in terms of accuracy of the predictions and computational efficiency is discussed.

Analysis of Dynamic Response of an MRI-Guided Magnetically-Actuated Steerable Catheter System.

This paper presents a free-space open-loop dynamic response analysis for an MRI-guided magnetically-actuated steerable intra-vascular catheter system. The catheter tip is embedded with a set of current carrying micro-coils. The catheter is directly actuated via the magnetic torques generated on these coils by the magnetic field of the magnetic resonance imaging (MRI) scanner. The relationship between the input current commands and catheter tip deflection angle presents an inherent nonlinearity in the proposed catheter system. The system nonlinearity is analyzed by utilizing a pendulum model. The pendulum model is used to describe the system nonlinearity and to perform an approximate input-output linearization. Then, a black-box system identification approach is performed for frequency response analysis of the linearized dynamics. The optimal estimated model is reduced by observing the modes and considering the Nyquist frequency of the camera system that is used to track the catheter motion. The reduced model is experimentally validated with 3D open-loop Cartesian free-space trajectories. This study paves the way for effective and accurate free-space closed-loop control of the robotic catheter with real-time feedback from MRI guidance in subsequent research.

Three-Dimensional Surgical Needle Localization and Tracking Using Stereo Endoscopic Image Streams.

This paper presents algorithms for three-dimensional tracking of surgical needles using the stereo endoscopic camera images obtained from the da Vinci ® Surgical Robotic System. The proposed method employs Bayesian state estimation, computer vision techniques, and robot kinematics. A virtual needle rendering procedure is implemented to create simulated images of the surgical needle under the da Vinci ® robot endoscope, which makes it possible to measure the similarity between the rendered needle image and the real needle. A particle filter algorithm using the mentioned techniques is then used for tracking the surgical needle. The performance of the tracking is experimentally evaluated using an actual da Vinci ® surgical robotic system and quantitatively validated in a ROS/Gazebo simulation thereof.

State Estimation for MRI-Actuated Catheters via Catadioptric Stereo Camera.

An MRI-actuated catheter is a novel robotic catheter system that utilizes the MR scanner for both remote steering and catheter tracking. In order to develop the mathematical model and the planning algorithm of the catheter in parallel to the MR tracking system, an alternative catheter tracking method is needed. This paper presents a catheter tracking algorithm based on the particle filter and the catadioptric camera system. The motion model of the particle filter is based on the quasi-static kinematics of the catheter. The measurement model calculates the weights of the particles according to the normalized cross-correlation of the segmented image from camera and a virtual rendering of the catheter. The efficacy of the tracking algorithm is demonstrates via experimental results.

Vision-Based Surgical Tool Pose Estimation for the da Vinci® Robotic Surgical System.

This paper presents an approach to surgical tool tracking using stereo vision for the da Vinci® Surgical Robotic System. The proposed method is based on robot kinematics, computer vision techniques and Bayesian state estimation. The proposed method employs a silhouette rendering algorithm to create virtual images of the surgical tool by generating the silhouette of the defined tool geometry under the da Vinci® robot endoscopes. The virtual rendering method provides the tool representation in image form, which makes it possible to measure the distance between the rendered tool and real tool from endoscopic stereo image streams. Particle Filter algorithm employing the virtual rendering method is then used for surgical tool tracking. The tracking performance is evaluated on an actual da Vinci® surgical robotic system and a ROS/Gazebo-based simulation of the da Vinci® system.

Experimental Validation of the Pseudo-Rigid-Body Model of the MRI-Actuated Catheter.

An MRI-actuated catheter is a novel robotic catheter system that utilizes the MRI for both remote steering and visualization for catheter ablation of atrial fibrillation. Planning and control of the catheter requires a sufficiently fast yet accurate model of the catheter. The pseudo-rigid-body (PRB) model offers a reasonable trade-off between speed and accuracy by approximating the continuum catheter as rigid links connected by flexible joints, thus reducing the infinite degrees of freedom of the continuum mechanism to a finite one. In this paper, a PRB model of the MRI-actuated catheter is validated experimentally by comparing the deflections of the PRB model with the deflections of the catheter prototype.

Iterative Jacobian-Based Inverse Kinematics and Open-Loop Control of an MRI-Guided Magnetically Actuated Steerable Catheter System.

This paper presents an iterative Jacobian-based inverse kinematics method for an MRI-guided magnetically-actuated steerable intravascular catheter system. The catheter is directly actuated by magnetic torques generated on a set of current-carrying micro-coils embedded on the catheter tip, by the magnetic field of the magnetic resonance imaging (MRI) scanner. The Jacobian matrix relating changes of the currents through the coils to changes of the tip position is derived using a three dimensional kinematic model of the catheter deflection. The inverse kinematics is numerically computed by iteratively applying the inverse of the Jacobian matrix. The damped least square method is implemented to avoid numerical instability issues that exist during the computation of the inverse of the Jacobian matrix. The performance of the proposed inverse kinematics approach is validated using a prototype of the robotic catheter by comparing the actual trajectories of the catheter tip obtained via open-loop control with the desired trajectories. The results of reproducibility and accuracy evaluations demonstrate that the proposed Jacobian-based inverse kinematics method can be used to actuate the catheter in open-loop to successfully perform complex ablation trajectories required in atrial fibrillation ablation procedures. This study paves the way for effective and accurate closed-loop control of the robotic catheter with real-time feedback from MRI guidance in subsequent research.

Acoustics of snoring and automatic snore sound detection in children.

OBJECTIVE: Acoustic analyses of snoring sounds have been used to objectively assess snoring and applied in various clinical problems for adult patients. Such studies require highly automatized tools to analyze the sound recordings of the whole night's sleep, in order to extract clinically relevant snore- related statistics. The existing techniques and software used for adults are not efficiently applicable to snoring sounds in children, basically because of different acoustic signal properties. In this paper, we present a broad range of acoustic characteristics of snoring sounds in children (N = 38) in comparison to adult (N = 30) patients. APPROACH: Acoustic characteristics of the signals were calculated, including frequency domain representations, spectrogram-based characteristics, spectral envelope analysis, formant structures and loudness of the snoring sounds. MAIN RESULTS: We observed significant differences in spectral features, formant structures and loudness of the snoring signals of children compared to adults that may arise from the diversity of the upper airway anatomy as the principal determinant of the snore sound generation mechanism. Furthermore, based on the specific audio features of snoring children, we proposed a novel algorithm for the automatic detection of snoring sounds from ambient acoustic data specifically in a pediatric population. The respiratory sounds were recorded using a pair of microphones and a multi-channel data acquisition system simultaneously with full-night polysomnography during sleep. Brief sound chunks of 0.5 s were classified as either belonging to a snoring event or not with a multi-layer perceptron, which was trained in a supervised fashion using stochastic gradient descent on a large hand-labeled dataset using frequency domain features. SIGNIFICANCE: The method proposed here has been used to extract snore-related statistics that can be calculated from the detected snore episodes for the whole night's sleep, including number of snore episodes (total snoring time), ratio of snore to whole sleep time, variation of snoring rate, regularity of snoring episodes in time and amplitude and snore loudness. These statistics will ultimately serve as a clinical tool providing information for the objective evaluation of snoring for several clinical applications.